Am I correct that the data hasn't changed (just converted from netcdf 4 to 3)?

I recall that last time we did this it appeared that it was BFB. Do we know if this is still true?

When we did something similar before, and went back later, it was easy to get confused. Specifically, the different file dates makes it easy to jump to the wrong conclusion, ie to mistakenly assume that the data was different. This is particularly likely to be an issue in this case as people will want to know how their configuration differs from the published CMIP6 simulation. Is there a way we can make it easy for users to figure out what is really going on?

Yes no data changes. This was confirmed by running cprnc for the 2 files.

Have we done this before? The new files were made a few months ago but AFAIK this is the first time use cases have been updated to use them.

Have we done this before? The new files were made a few months ago but AFAIK this is the first time use cases have been updated to use them.

Not for these specific files, but for other files. See PR #2174, issue #1970, and several other related issues.

One of the most logical things to do (IMHO) when trying to do work based on an old configuration with a new code base is to build the same compset and check that the namelists are the same. This change will produce many differences, and the user will need to figure out what has changed for every single file, or ignore the differences and potentially miss some other important change.

last time, I recall, a comment was added to the metadata in the new files, which is a help. Did that happen here too?

Is there anything else we can do to avoid future confusion?

Yes there is metadata in each file clearly indicating that it was only a format change.

So PR #2174 changed these same files for 1950 on master. This PR changes them for 1850 and 20TR on maint-1.0 and eventually master. But the 1950 change uses files that end in c180205. If that was just a format change, we should use those instead of the c181106 ones to further reduce confusion.

So PR #2174 changed these same files for 1950 on master. This PR changes them for 1850 and 20TR on maint-1.0 and eventually master. But the 1950 change uses files that end in c180205. If that was just a format change, we should use those instead of the c181106 ones to further reduce confusion.

That is my recollection, and if we can use those previous files that would certainly be an improvement.

I am pinging @PeterCaldwell and @golaz to make sure I am remembering correctly.

BTW, if we can use the files from PR 2174, then can we delete the newer ones from the data server to reduce confusion even further?

Now I'm having second thoughts. There is more and better metadata in the c181106 files. Compare the global attributes for each
c180205:

	:history = "Created on 05/25/2017 at Pacific Northwest National Lab. Contact: hailong.wang@pnnl.gov\n",
			"Converted from netcdf-4 to netcdf-3 by P. Caldwell using netcdf4_to_netcdf3.py on 2018-02-05\n",
			"" ;

c181106:'

		:history = "Tue Nov  6 10:42:34 2018: ncks -O -3 --glb E3SM_input_conversion=20181106: Format converted from netcdf4-classic to classic (netCDF3) for compatibility with PnetCDF reader. No data were changed /lcrc/group/acme/public_html/inputdata/atm/cam/chem/trop_mozart_aero/emis/DECK_ne30/cmip6_mam4_bc_a4_elev_1850-2014_c170525.nc inputdata/atm/cam/chem/trop_mozart_aero/emis/DECK_ne30/cmip6_mam4_bc_a4_elev_1850-2014_c181106.nc\n",
			"Created on 05/25/2017 at Pacific Northwest National Lab. Contact: hailong.wang@pnnl.gov" ;
		:E3SM_input_conversion = "20181106: Format converted from netcdf4-classic to classic (netCDF3) for compatibility with PnetCDF reader. No data were changed" ;
		:NCO = "4.6.7" ;

The c181106 files include the name of the file that was converted and adds 2 new global attributes.

I agree that the new metadata is better.  I am not sure, though, whether this outweighs the confusion from an extra set of files.  If you do choose to go with the new files, can you also update the other compsets that use the 180205 files?

The 180205 files have been used in code that was checked in. So we can't delete them. Might as well just continue to use them instead of the never-used 181106 ones.

Is there a list somewhere of all the files @PeterCaldwell converted?

We shouldn't delete the 180205 files. But we could modify master so that the compsets switch to using the newer netcdf files.

I recall that @PeterCaldwell had a script for automatically finding and converting the netcdf4 files, and it would spit out the files it converted. Would it be easier just to search the data archives for '180205'?

Closing this. A new PR will replace it.